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K-12 Education: Education Should Take Immediate Action to Address Inaccuracies in 
Federal Restraint and Seclusion Data 


As we reported in February 2019, the Department of Education’s (Education) data suggest that 
the restraint and seclusion of K-12 public school students is rare nationwide, though it 
disproportionately affects students with disabilities and boys in general.' In broad terms, 
Education defines restraint as restricting a student’s ability to freely move his or her torso, arms, 
legs, or head, and defines seclusion as involuntarily confining a student alone in a room or area 
from which the student is physically prevented from leaving. Education’s 2012 resource 
document on the use of restraint and seclusion states that restraint or seclusion should never be 
used except when a child’s behavior poses imminent danger of serious physical harm to self or 
others.” 


Every 2 years, Education collects and publicly reports data from nearly all public school districts 
and schools as part of its Civil Rights Data Collection (CRDC).° Districts self-report and certify 


1GAO, K-12 Education: Federal Data and Resources on Restraint and Seclusion, GAO-19-418T (Washington, D.C.: 
Feb. 27, 2019). 


*U.S. Department of Education, Restraint and Seclusion: Resource Document (Washington, D.C.: May 15, 2012). 
According to Education, this resource document is intended to serve as a resource for states, localities, and districts 
to consider when developing policies and procedures on restraint and seclusion. 


5Except for Puerto Rico, districts in US territories are not required to participate in the CRDC. Similarly, districts are 
not required to provide information for tribal schools operated by the Department of the Interior's Bureau of Indian 
Education. Schools operated by the Department of Defense Education Activity are also not required to participate, 
according to Education. 
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the data. Education’s Office for Civil Rights (OCR) uses CRDC data in its enforcement of 
various federal civil rights laws prohibiting discrimination on the basis of race, color, national 
origin, sex, and disability. 


We have work under way on districts’ reporting practices for restraint and seclusion data in 
response to a provision in the explanatory statement from the House Committee on 
Appropriations accompanying the Consolidated Appropriations Act of 2018. As part of our data 
reliability testing for that work, we analyzed the number of districts that left fields pertaining to 
restraint and seclusion blank, or that reported all zeros for those fields, to determine the 
prevalence of blanks or zeros in the CRDC at the national, state, and district levels. Our data 
reliability testing raised questions about the completeness and accuracy of the CRDC restraint 
and seclusion data. We are therefore issuing this separate report on the issues we have 
identified to date regarding potentially incomplete data. Because Education is currently 
collecting and validating restraint and seclusion data for the 2017-18 school year, it is important 
it take immediate steps to address underreporting before it publishes these data.* 


As part of this work, we reviewed the explanations that Education requires the largest districts to 
provide if they report zero incidents of restraint and seclusion. We also reviewed documentation 
on Education’s investigations into underreporting of restraint and seclusion. Additionally, we 
interviewed federal Education officials and the contractor responsible for maintaining the CRDC 
and providing routine CRDC technical assistance, as directed by OCR. We conducted this 
performance audit from March 2019 to June 2019 in accordance with generally accepted 
government auditing standards. Those standards require that we plan and perform the audit to 
obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence obtained provides a 
reasonable basis for our findings and conclusions based on our audit objectives. 


Background 


Education requires nearly every school district in the country to submit data on their schools to 
Education’s CRDC through an online submission tool.° This tool automatically performs edit 
checks, which flag data errors or potential errors. These edit checks are called “business rules” 
and occur in real time as districts enter data or after they upload files. 


The CRDC online tool uses three business rules specifically related to restraint and seclusion 
data. If the data entered by a district triggers any of these rules, an error message appears with 
instructions on how to clear the errors. One rule triggers an error message if a district with more 
than 100,000 students enrolled reports zero incidents of restraint or zero incidents of seclusion.® 
In part, the error message states that, “based on trends in past data, at least one instance of 
restraint or seclusion occurs within a population greater than 100,000 students.” The online tool 
then prompts the district to correct the error. If the district finds no error, it is instructed to explain 


4According to the Department of Education, as of May 22, 2019, 94 percent of districts had submitted their 2017-18 
school year data. 


“There are two options for data submission. Districts may enter data through data entry screens or through a data file 
upload. 


SFor 2017-18, the threshold was lowered so that the rule applied to districts with more than 50,000 students enrolled. 
The other two rules are logic tests that compare the number of incidents to the number of students subjected to 
restraint and seclusion. 
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why the data it submitted were valid. For districts of any size, if data are missing, districts are 
required to provide an explanation and submit an action plan for reporting the required data in 
the next CRDC. ’ 


Districts must certify that the data they submit are “true and correct,” which is a step completed 
by the district superintendent or an authorized designee. The CRDC submission tool should not 
allow a district to certify its submission unless all required data pass the system validation 
checks or all errors are explained. If districts have not collected the data required for the 
CRDC—or if the data are unavailable for some other reason—districts are to leave relevant data 
cells blank. A zero in a data cell should represent an actual count—that is, zero students were 
restrained or secluded. 


Education’s contractor, working in conjunction with OCR, conducted the 2015-16 CRDC and is 
currently coordinating the 2017-18 CRDC. At OCR’s direction, the contractor manages all 
aspects of the CRDC, including the online submission tool, data quality, and data analysis. The 
contractor also provides routine technical assistance by responding to requests from districts 
that need help. 


In January 2019, Education officials announced an initiative to address inappropriate use of 
restraint and seclusion. According to Education officials, OCR will conduct data quality reviews 
of the CRDC in four to five school districts in each of OCR’s 12 regions, and provide technical 
assistance to schools to ensure districts are collecting and reporting accurate restraint and 
seclusion data. 


CRDC Data Do Not Reflect All Incidents of Restraint and Seclusion 


For the most recent CRDC—school year 2015-16—70 percent of the more than 17,000 school 
districts in the U.S. reported zero incidents of restraint and zero incidents of seclusion. In 39 
states and the District of Columbia, more than half of the school districts reported zeros; and in 
12 states, 80 percent or more of the districts reported zeros. (See fig. 1.) However, our analyses 
of 2015-16 CRDC data and review of Education documents indicate that CRDC data do not 
accurately capture all incidents of restraint and seclusion in schools. 


7 According to the manual for the public use file for the 2015-16 CRDC, to indicate the reason why a district was able 
to certify its submission with a blank, Education’s contractor later adds a “reserve code” to the blank field. For 
example, one code indicates that although the data are missing, the district was able to certify its submission 
because it submitted an action plan. Other codes indicate that the field is blank because of a system error or because 
the district was able to skip questions that did not apply to that district. 
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I 
Figure 1: Percentage of K-12 Public School Districts Reporting Zeros Related to Restraint or Seclusion, by 
State, School Year 2015-16 
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Note: At the time of our review, the latest data available were from the 2015-16 school year. Restraint includes both physical 
restraint and mechanical restraint. Hawaii has only one school district. 


While according to Education’s business rule districts with more than 100,000 enrolled students 
are likely to have at least one incident of restraint or seclusion, one-third (10 of 30) of such 
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districts in the country reported zeros in the 2015-16 school year.® (See table 1.) 


Table 1: Restraint and Seclusion Data for K-12 Public School Districts with More than 100,000 Students, 
School Year 2015-16 


Size Number of Number of Incidents of Incidents of 
District State Rank Schools Students Restraint Seclusion 
NEW YORK CITY 
PUBLIC 
SCHOOLS NY 1 1,637 984,500 0? 0 
LOS ANGELES 
UNIFIED CA 2 785 539,634 108 0 
CITY OF CHICAGO 
SCHOOL DISTRICT IL 3 579 392,303 47 6 
DADE FL 4 485 358,179 0 0 
CLARK COUNTY 
SCHOOL 
DISTRICT NV 5 361 326,238 2,603 0 
BROWARD FL 6 348 269,502 0 0 
HOUSTON 
INDEPENDENT 
SCHOOL DISTRICT TX 7 282 215,989 58 0 
HILLSBOROUGH FL 8 289 211,731 17 21 
ORANGE FL 9 243 196,987 al 82 
PALM BEACH FL 10 234 188,590 91 0 
FAIRFAX COUNTY 
PUBLIC SCHOOLS VA 11 206 185,630 0 0 
HAWAI| DEPARTMENT 
OF 
EDUCATION HI 12 290 182,913 0 0 
GWINNETT COUNTY GA 13 135 175,958 1,150 0 
WAKE COUNTY 
SCHOOLS NC 14 171 159,149 0 0 
DALLAS 
INDEPENDENT 
SCHOOL DISTRICT TX 15 239 158,941 117 0 
MONTGOMERY 
COUNTY 
PUBLIC SCHOOLS MD 16 203 156,819 1,078 337 
CHARLOTTE- 
MECKLENBURG 
SCHOOLS NC 47 168 149,270 9 0 


ss Philadelphia City, PA and Prince George’s County, MD reported zero students subjected to restraint or seclusion, 
but left the number of incidents blank, meaning they do not collect the data. 
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Size Number of Number of Incidents of Incidents of 
District State Rank Schools Students Restraint Seclusion 
PRINCE GEORGE'S 
COUNTY 
PUBLIC SCHOOLS MD 18 208 142,536 i : 
PHILADELPHIA CITY 
SCHOOL DISTRICT PA 19 218 133,814 : 
SAN DIEGO UNIFIED CA 20 226 130,964 188 35 
DUVAL FL 21 206 129,003 34 24 
CYPRESS- 
FAIRBANKS ISD T™ 22 83 113,912 76 0 
SHELBY COUNTY 
SCHOOLS TN 23 207 113,208 248 77 
COBB COUNTY GA 24 114 112,708 149 8 
BALTIMORE COUNTY 
PUBLIC SCHOOLS MD 25 169 110,786 880 365 
NORTHSIDE 
INDEPENDENT 
SCHOOL DISTRICT TX 26 116 104,847 0 0 
PINELLAS FL 27 155 102,893 36 39 
POLK FL 28 152 101,468 0 0 
DEKALB COUNTY GA 29 133 101,355 3 0 
JEFFERSON COUNTY KY 30 172 101,018 4,134 257 


Source: GAO analysis of 2015-2016 CRDC data. | GAO-19-551R 


Note: At the time of our review, the latest data available were from the 2015-16 school year. Restraint totals include both physical 
restraints and mechanical restraints. Districts that reported no incidents are shaded gray. 


*New York City’s data on mechanical restraint were missing. The district reported zero incidents of physical restraint. 


Because Prince George’s County, MD and Philadelphia City, PA reported that zero students were subjected to restraint or 
seclusion while leaving the number of incidents blank, we included them among the districts reporting zeros. 


When school districts with more than 100,000 enrolled students reported zero incidents to 
CRDC, they were to correct the apparent error by modifying the data or confirming that the data 
were valid. If data were unavailable, they were to leave the fields blank and submit an action 
plan to explain how they would collect the data in the future. Districts were to report zeros only 
to indicate that there were no incidents of restraint or seclusion. However, only one of the 10 
districts with more than 100,000 enrolled students that reported zeros, Hawaii Department of 
Education, reported to Education that the zeros actually represented zero incidents. The other 
nine districts reported zeros but, according to explanations provided to Education or information 
districts provided publicly, had incidents they did not report, had incidents they were unable to 
report, or were not collecting the data. Specifically: 


e Northside Independent School District in Texas stated that it did not have a way to 


export its data to the CRDC but could provide the data upon request. Wake County 
Public School System in North Carolina indicated that it was unable to disaggregate its 
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restraint and seclusion data by school, race, and gender, as required by the CRDC. 
Further, neither Northside nor Wake submitted an action plan, as required by Education, 
to indicate how they would collect and report these data in the future. These districts 
were able to bypass the CRDC system requirement to provide an action plan because 
they reported zero incidents instead of leaving the fields blank to indicate the data were 
missing. 


e New York City, NY; Philadelphia, PA; and Prince George’s County, MD were not 
collecting data on restraint and seclusion and should have left the fields blank in the 
CRDC submission tool. These districts did submit action plans for reporting the data in 
the future, as required for missing data. However, New York City’s action plan only 
discussed the reasons why it was unable to collect and report data on mechanical 
restraint. It did not mention collecting the remaining data on physical restraint or 
seclusion, nor affirm that the zeros were correct. According to Education officials, they 
did not request that New York City provide a plan for collecting the remaining data. In 
commenting on a draft of this report, Education stated prior to the 2017-18 submission 
that it emailed districts with action plans and requested that the districts review their own 
action plans. 


e Fairfax County, VA reported zero incidents of both restraint and seclusion but, according 
to Education officials, due to a system error, the CRDC business rule was not triggered. 
Fairfax did not provide an action plan explaining how it would meet reporting 
requirements in the next collection, according to Education officials. Fairfax County 
Public Schools have since publicly stated that they did, in fact, have incidents, and in 
school year 2017-18, had more than 1,600 incidents. 


e Restraint and seclusion data for the remaining three districts, all in Florida, were 
submitted by the state, according to Education officials. The state reported zero 
incidents for all three districts. While Florida submitted an action plan for some CRDC 
data elements, the plan did not address the reports of zero incidents of restraint and 
seclusion. According to Education officials, a submission system error occurred allowing 
Florida to certify without including the restraint and seclusion data elements in their 
action plan. Education did not ask Florida to submit a revised plan. 


For these nine districts, the CRDC’s 2015-16 publicly available data file remains uncorrected. 
Specifically, the CRDC still indicates that the districts had zeros related to restraints or 
seclusions, rather than indicating that the districts did not report data. While it is difficult to know 
the full extent of underreporting of restraint and seclusion in the CRDC, the fact that only one of 
the 10 largest districts that reported zeros actually affirmed that it had no incidents calls into 
question the data showing zero incidents for 70 percent of the nation’s public school districts. 


For the current collection, Education is applying the business rule for reporting zero incidents of 
restraint and seclusion to more districts, but it is unclear if this change will significantly improve 
reporting. Education officials said that they were concerned that the 2015-16 collection rule for 
districts with more than 100,000 enrolled students, which included 30 districts or 13 percent of 
total public school enrollment, did not adequately help them detect reporting problems. 
Therefore, in the 2017-18 data collection, they are applying the rule to districts with more than 


9 Florida submits and certifies data on behalf of all school districts. 


Page 7 GAO-19-551R Accuracy of Restraint and Seclusion Data 


50,000 enrolled students. Education officials were unable to provide us with any analysis used 
to establish either the 100,000 or 50,000-student threshold for the CRDC business rule. We 
determined that, had this change been in effect for the 2015-16 CRDC, Education’s business 
rule would have applied to just 65 more districts, or to 21 percent of total public school 
enrollment. However, the rule still does not apply to the overwhelming majority of school districts 
that report zero incidents of restraint or seclusion, leaving the meaning of zeros unclear for 
smaller districts attended by about 80 percent of public schoolchildren. 


Documents from Education that we reviewed indicated that the misreporting of zeros occurred 
in smaller districts as well. These erroneous reports were detected either by the media or in the 
course of larger investigations by Education into restraint and seclusion, rather than by any 
systemic review of the CRDC data. Specifically: 


e After Education’s Office of Inspector General received a congressional inquiry in 2018, 
Education followed up with three smaller districts in lowa that reported zero incidents of 
restraint and seclusion, according to documents provided by Education. 


e Documentation on Education’s website showed that Education found underreporting in 
previous CRDC collections. Specifically, from 2014 to 2017, Education completed 
investigations on the inappropriate and disproportionate use of restraint and seclusion 
and found underreporting in four districts in Texas, Utah, Virginia, and California. Two of 
these districts reported zero incidents of restraint and zero incidents of seclusion to the 
CRDC, while Education’s investigations revealed that incidents had occurred. A third 
district reported zero incidents for students without disabilities, stating that no data were 
maintained for those students, although the district had reported incidents in earlier 
collection periods. 


e CRDC technical data notes accompanying the results of all four CRDC surveys since 
2009 (when Education began collecting data on restraint and seclusion) cautioned that 
districts might have reported zero incidents, instead of indicating that the information 
was not available. Further, the 2015-16 notes said that districts with incomplete data 
may have reported zero incidents, and that it is not possible to determine where this may 
have occurred. 


Collecting accurate civil rights data is key to OCR’s mission to ensure equal access to education 
and to promote educational excellence throughout the nation through vigorous enforcement of 
civil rights laws that protect students from discrimination on the basis of race, color, national 
origin, sex, or disability. Federal Standards for Internal Control state that agency management 
should use quality information to achieve the entity’s objectives. These standards also note that 
such data should be reasonably free from error and bias and faithfully represent what they 
purport to represent, and that agency management should evaluate sources of data for 
reliability. '° Absent reliable and accurate data, neither Education nor the public can know the 
prevalence of restraint and seclusion in public schools. 


1°GA0, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 
2014). 
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Instructions for Reporting Zero Incidents versus Reporting Unavailable Data Are Difficult 
to Find 


Education created a CRDC data tip sheet clarifying when school districts should report zero 
incidents of restraint and seclusion and how to indicate that data are not available. However, the 
tip sheet may be difficult for districts to find because it is not on the data entry screen itself, nor 
in the instructions for submitting data files. Instead, the tip sheet is located on a CRDC technical 
assistance website. '' Further, we were only able to find the tip sheet after searching page by 
page on the website because the search function was difficult for us to find.'* According to 
Education officials, within the CRDC data tool, there is a link to a list of resources. The tip sheet 
is on that list of resources, but its instructions are not visible to the person entering the data. 


So that external parties can help an agency achieve its objectives and address related risks, 
Federal Standards for Internal Control state that agencies should communicate information 
relating to activities that impact the internal control system. Agency management should 
periodically evaluate its methods of communication and consider a variety of factors in selecting 
an appropriate method of communication. One such factor to consider is availability. These 
standards state that information should be readily available to the audience when needed. "? 
Absent easily accessible information about how to report accurate data, school districts may 
continue to erroneously report zero incidents when data are in fact not available, misleading the 
public and policymakers about the prevalence of restraint and seclusion in public schools. 


Conclusions 


Our analyses raise questions about whether the confirmed instances of misreported zeros to the 
CRDC are indicative of a more pervasive pattern of underreporting of restraint and seclusion in 
U.S. public schools. Although districts, not Education officials, are required to certify their data 
as “true and correct,” Education has repeatedly published restraint and seclusion data in its 
CRDC without always correcting known reporting errors. Data left uncorrected could further 
undermine the public’s confidence in these data and limit the utility of a dataset intended to 
assist with federal civil rights monitoring, enforcement, and oversight. We believe the 2017-18 
data collection is at risk for similar reporting issues. 


A fundamental first step toward improving the quality of the restraint and seclusion data is to 
assure that when school districts report zero incidents it truly means there were no incidents, 
and to accurately distinguish districts with no incidents from districts that do not track or collect 
the data. Given that Education is currently collecting restraint and seclusion data for the 2017- 
18 school year, it is important that Education immediately take steps to address underreporting. 
Failure to do so will result in data that continues to provide an incomplete picture of the 


"The technical assistance website can be found at https://crdc.grads360.org/#program. There is a link to a resources 
page with the data tip sheet on it within the submission system, but the link does not appear on the data entry screen. 
In commenting on this report, Education stated that the instructions are available in a new user starter kit. We found 
that the data tip sheet is referenced but the instructions on the sheet are not included in the kit. 


12 On the website homepage, we found links to five main resource pages with information on preparing and 
submitting CRDC data. In a separate text box near the bottom of the homepage, we found links to six additional 
webpages. One of those webpages on “Data Tips” contained the clarification. The clarification did not appear on the 
data entry screen itself or in the instructions for submitting files. 


13GA0-14-704G. 
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prevalence of restraint and seclusion, leaving OCR unable to reliably use a key tool in carrying 
out its enforcement of civil rights laws. 


Recommendations 
GAO is making four recommendations to the Department of Education’s Office for Civil Rights: 


The Assistant Secretary for the Office for Civil Rights should immediately remind and clarify for 
all school districts that they are to only report zero incidents of restraint and seclusion when 
there are none and that they are to leave cells blank to indicate when data are not collected or 
incomplete. The Assistant Secretary should also ensure that instructions for when to record 
zeros and when to leave cells blank are prominently displayed and readily available to districts 
as they complete the CRDC. (Recommendation 1) 


The Assistant Secretary for the Office for Civil Rights should, as part of the 2017-18 CRDC 
quality assurance process, follow up with school districts that have already submitted reports of 
zero incidents of restraint or seclusion to obtain assurances that zero incidents means no 
incidents or ask the districts to submit corrected data. (Recommendation 2) 


The Assistant Secretary for the Office for Civil Rights should monitor compliance with its action 
plan requirement, and ensure plans are submitted and address all missing data. 
(Recommendation 3) 


The Assistant Secretary for the Office for Civil Rights should prominently disclose for past 
collections the potential problems with using restraint and seclusion data given the known 
misreporting issues, such as those detailed in this report involving 9 of the nation’s large public 
school districts. (Recommendation 4) 


Agency Comments and Our Evaluation 


We provided a draft of this report to the Department of Education for review and comment. 
Their comments are reproduced in enclosure |. Education agreed with 3 of our 4 
recommendations, but provided the caveats that implementation of the recommendations was 
contingent on the availability of resources, and that the 2017-18 collection was too far along to 
clarify instructions. However these data have not been published and Education allows districts 
a significant period of time in which to correct errors. Education disagreed with our fourth 
recommendation that they correct known errors in previous reporting. Education also provided 
technical comments, which we incorporated as appropriate. 


With respect to our first recommendation, Education agreed to remind districts going forward 
that they are only to report zero incidents of restraint and seclusion when there are none and to 
ensure that instructions for when to record zeros and when to leave cells blank are prominently 
displayed, but said that this reminder is too late for the 2017-18 data collection because 94 
percent of districts had already submitted their data. Dependent on available funds and staff 
resources, Education agreed to feature the instructions more prominently on the website and 
consider other changes, such as targeted communications and changes in the placement of the 
instructions for the 2019-20 collection. We are encouraged that Education recognizes the 
seriousness of this issue and the data quality issues it has allowed to persist when districts 
inappropriately and inaccurately report zero incidents of restraint and seclusion. While we 
appreciate that most districts have already submitted data for 2017-18, Education allows 
districts a significant period of time in which to correct errors. We believe that reminding districts 
about when to record zeros, even retroactively, could prompt districts to address any errors 
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before Education publishes these data sometime in 2020, thereby improving the integrity of its 
CRDC data. We urge Education to remind districts for the current 2017-18 collection about its 
requirements to only report zero incidents of restraint and seclusion when there are none and 
when to leave cells blank for the current collection and subsequent collections. 


With respect to our second recommendation, Education said it agreed with the overall intent of 
following up with districts that have already submitted reports of zero incidents to obtain 
assurances that zero incidents means no incidents. Education stated that once the 2017-18 
collection ends, it will review data quality findings, including the reporting of zero incidents, to 
determine appropriate outreach to states and districts. Education stated depending on available 
resources, it would also continue to explore opportunities to follow up on submissions of reports 
of zero incidents. We continue to urge Education to make follow-up a priority before Education 
makes these data publicly available so that the public, researchers, and federal policymakers 
that use these data know if it can be relied upon. 


With respect to the third recommendation, Education agreed to monitor compliance with its 
action plan requirement. Education stated that it had recently taken steps to more closely 
scrutinize action plans for the 2017-18 data collection, including directly communicating with 
districts about their action plans and scheduling calls with any district that requests similar or 
repetitious action plans over the course of two or more collections. 


Regarding the fourth recommendation that Education correct known errors in previous CRDC 
data collections, Education disagreed, saying it did not believe it feasible to continually update 
the published data files for closed collections due to the need to finalize consistent data for the 
federal agencies, policymakers, researchers, educators, school officials, and others who use the 
CRDC data. Instead, Education stated that for the 2015-16 data collection, it would amend the 
data notes to ensure the public is aware of potential changes. We appreciate Education’s 
concern for its resources. However, as Education pointed out, these data are widely used by 
policymakers, researchers, educators, school officials, and others. Because there is a 
compelling reason to believe that there is some misreporting of restraint and seclusion data, and 
because Federal Standards for Internal Control provide for agencies to use quality information 
to achieve the entity's objectives, it is important that, at a minimum, Education be clear and 
transparent about the limitations of the restraint and seclusion data it makes publicly available. 
We have therefore revised our recommendation to state that for past collections Education 
should prominently disclose the potential problems with using restraint and seclusion data given 
the known misreporting issues, such as those detailed in this report involving 9 of the nation’s 
largest public school districts. 


We are sending copies of this report to the Secretary of Education, appropriate congressional 
committees, and other interested parties. In addition, the report will be available at no charge on 
the GAO website at http://www.gao.gov. 


If you and your staff have any questions, please contact Jacqueline M. Nowicki at (617) 788- 
0580 or nowickij@gao.gov. Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. GAO staff who made major contributions to 
this report were Sherri Doughty (Assistant Director), Lara Laufer (Analyst-in-Charge), James 
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Bennett, Holly Dye, Maria Gadel, Morgan Jones, Christy Ley, Jean McSween, John Mingus, 
Kristin Petroff, and James Rebbe. Also contributing were Deborah Bland and Sheila McCoy. 


Jacqueline M. Nowicki 
Director, Education, Workforce, and Income Security 


Enclosure 
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Enclosure 1: Comments from the Department of Education 


UNITED STATES DEPARTMENT OF EDUCATION 
OFFICE FOR CIVIL RIGHTS 


THE ASSISTANT SECRETARY 


May 24, 2019 


Ms. Jacqueline Nowicki, Director 
Education, Workforce, 

and Income Security Issues 
Government Accountability Office 
Washington, D.C. 20548 


Dear Ms. Nowicki: 


Thank you for the opportunity to provide comments on the Government Accountability Office’s (GAO) 
draft report, K-12 Education: Education Should Take Immediate Action to Address Inaccuracies in Federal 
Restraint and Seclusion Data (GAQ-19-551R). Iam providing comments on behalf of the U.S. Department 
of Education (Department) regarding the Office for Civil Rights’ (OCR) Civil Rights Data Collection 
(CRDC). I am also attaching to this letter a separate document detailing technical edits for GAO’s 
consideration. 


As a threshold matter, the Department shares GAO’s concern with the issue of restraint and seclusion 
practices in our nation’s schools. Indeed, on January 17, 2019, Secretary Betsy DeVos announced the 
Department’s initiative on the inappropriate use of restraint and seclusion, which is overseen by OCR, in 
partnership with the Office of Special Education and Rehabilitative Services (OSERS).' As part of the 
initiative, OCR has made restraint and seclusion the focus of its primary proactive compliance initiative for 
this fiscal year, independent of GAO’s review. OCR’s current focus on restraint and seclusion is broader 
than GAO’s review, since it includes not only improvement of data issues but also compliance with federal 
disabilities laws. We agree with GAO that data quality and accuracy are very important, particularly on an 
issue like restraint and seclusion, and because accurate reporting may assist in ensuring appropriate use of 
behavioral strategies and tools given the possibility of abuse that could arise if such tools are used 
indiscriminately. 


As you know, the CRDC is a biennial survey of public schools and school districts in the United States. 
The CRDC measures student access to courses, programs, staff, and resources that relate to OCR’s 
jurisdiction.? The CRDC also is a resource for other federal agencies, policymakers, researchers, educators, 


' One part of the initiative includes outreach to selected LEAs with restraint and seclusion data anomalies — to ensure 
data submitted are true and correct. See hittps://Avww.ed.gov/news/press-releases/us-department-education- 
announces-initiative-address-inappropriate-use-restraint-and-seclusion-protect-children-disabilities-ensure- 
compliance-federal-laws. 

? Through the CRDC, OCR collects information from local educational agencies (LEAs) that receive financial 
assistance from the Department, including public school districts, charter schools, juvenile justice facilities, alternative 
schools, and schools serving only students with disabilities. This includes LEAs in all 50 states, the District of 
Columbia, and commencing with the 2017-18 collection, the Commonwealth of Puerto Rico. In addition to reporting 
by LEAs, some State education agencies (SEAs) assist their LEAs in CRDC reporting. Other U.S. territories are not 
currently required to participate in the CRDC. Likewise, LEAs are not required to provide information for the CRDC 
for tribal schools operated by the U.S. Department of the Interior’s Bureau of Indian Education, nor are schools 
operated by the U.S. Department of Defense Education Activity required to participate in the CRDC. In addition, if 
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school officials, parents/guardians, students, other stakeholders, and members of the public. We strive for 
State education agencies (SEAs) and local educational agencies (LEAs) to report data accurately and 
completely. 


Again, the Department appreciates GAO examining the quality of the data that the CRDC receives from 
SEAs and LEAs, and GAO’s understanding that all data received by the Department are self-reported, and 
self-certified by the Department’s recipients as true and correct by cach Superintendent or his or her 
designee. Because GAO’s draft report focuses on ways to improve the CRDC’s data quality by examining 
the 2015-2016 CRDC data, I am highlighting several areas of concern, with the GAO analysis, that we 
recommend be addressed to achieve this purpose. The Department’s responses to GAO’s draft report are 
below. 


OCR’s Amended Business Rule for Restraint and/or Seclusion 


Before responding to GAO’s recommendations, OCR would first like to address GAO’s questions about 
OCR’s current business rule, and how the rule applies to LEAs with 50,000 or greater in student 
enrollment. OCR would like to point out that the present aim of the business rule is to focus on a subset 
of LEAs with potentially problematic data submissions, as opposed to creating a rule that triggers an error 
message for most LEAs. However, OCR will evaluate the effectiveness of this rule in the 2017-18 
collection, to determine if any adjustments will be recommended for the 2019-20 collection* 


GAO’s First Recommendation: 


GAO states that the Assistant Secretary for Civil Rights should immediately “remind and clarify for all 
districts that they are to only report zero incidents of restraint and seclusion when there are none and that 
they are to leave cells blank to indicate when data are not collected or incomplete,” and “ensure that 
instructions for when to record zeros and when to leave cells blank are prominently displayed and readily 
available to districts as they complete the CRDC.” 


OCR agrees that it is important that LEAs understand the significance of reporting a “zero” figure, or 
alternatively leaving a data entry cell blank before obtaining an action plan to address a missing data 
element. OCR also agrees that it is important to ensure that LEAs can access information that is 
prominently displayed during the CRDC submission process. As GAO is aware, instructions are currently 
available on the CRDC’s New User Starter Kit, on page 14, on the CRDC technical assistance website, 
under “data tips,” and finally, with the data tip links that are accessible on the CRDC on-line submission 
tool. 


a public school’s enrollment for a school year was less than 5 students for fewer than 60 days, OCR may exempt that 
school from CRDC reporting. 

3 GAO notes that OCR’s business rule “still does not apply to the overwhelming majority of school districts that report 
zero incidents of restraint and seclusion, leaving the meaning of zeroes unclear for smaller districts attended by about 
80 percent of public schoolchildren.” 

4 A business rule regarding restraint and/or seclusion data was first created for use during the 2013-14 CRDC based 
on analyses and reviews of the 2011-12 CRDC data following data quality reviews of the 2011-12 CRDC data. 
Through the data quality reviews, OCR became aware of the need to set “threshold” business rules for a few data 
elements because of issues raised about whether the largest districts were able to complete the collection by reporting 
zeros. For restraint and/or seclusion, the data analyses resulted in OCR’s decision to implement the business rule 
focused on what level of student enrollment was appropriate for a threshold. It was established for LEAs with 
enrollments of 100,000 or more students, which would identify possible misreporting for the largest student 
populations. For the 2017-18 CRDC, OCR changed the business rule’s threshold from 100,000 or more to 50,000 or 
more students. 
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However, to the extent that confusion for some LEAs remains, OCR will feature instructions more 
prominently on the website, and will make changes, dependent on available funds and staff resources. It is 
clear that GAO would like these changes to be made for the 2017-18 collection, but OCR disagrees as to 
the timing of the recommended changes, considering that more than 94 percent of all LEAs already have 
completed data submissions. In the interest of providing LEAs notice of the data collection changes, OCR 
will consider the changes for the 2019-20 collection. Such changes could include the placement of notices 
on appropriate websites, augmenting the submission tool so that an instruction message appears before data 
is entered, targeted LEA communications, and including this information in the letter I send to LEAs 
announcing the opening of the 2019-20 collection. 


GAO’s Second Recommendation: 


GAO states that the Assistant Secretary for OCR “should, as part of the 2017-18 CRDC quality assurance 
process, follow up with districts that have already submitted reports of zero incidents of restraint and 
seclusion to obtain assurances that zero incidents means no incidents or ask the districts to resubmit 
corrected data.” 


Once the 2017-18 collection ends, and the data quality assurance process is finalized, OCR will review data 
quality findings, including the reporting of zero incidents of restraint and seclusion, to determine 
appropriate outreach to SEAs and LEAs.> Such outreach may not necessarily include a request for an 
additional signed assurance, since LEAs already currently certify that all submitted data submitted are true 
and correct. Nevertheless, OCR agrees with the overall intent of the recommendation, which is to 
accurately reflect the reporting of zero incidents. Depending on available resources, OCR will continue to 
explore opportunities to follow up on LEAs’ submissions of reports of zero incidents of restraint and 
seclusion for the 2017-2018 collection and for the 2019-2020 collection. Of course, it must be noted that 
the submission of zeros is not always indicative of incorrect/incomplete data. 


GAO’s Third Recommendation 


GAO states that the Assistant Secretary for OCR “should monitor compliance with its action plan 
requirement, and ensure plans are submitted and address all missing data.” 


OCR agrees with this recommendation, and already has taken steps to more closely scrutinize action plans 
for the 2017-2018 data collection.’ The additional safeguards put into place include: directly 
communicating with each LEA with an approved action plan to seek confirmation that the LEA will take 
the needed steps to collect and report the data for the 2019-20 collection (to take place at the close of the 
2017-18 collection and in advance of the 2019-20 collection); rejecting all 2017-18 action plans that lack a 
detailed timeline explaining the steps that will be taken to collect missing data in time for the 2019-20 


> OCR has an established system in place to promptly address data anomalies and any requests to correct data after 
the data have been released. When a district believes that data submitted to OCR may be inaccurate, the Department 
encourages districts to contact OCR directly at ocrdata@ed.gov. If OCR receives a data correction request a year or 
more after a collection closes, OCR will include a data note in the file to alert potential data users. 

6 The current certification reads: “I am authorized to submit the LEA’s forms for certification because I am the LEA 
Superintendent [or designated by the LEA Superintendent to certify]. I certify that the information provided is true 
and correct to the best of my knowledge and belief. A willfully false statement is punishable by law. (18 U.S.C. 
§ 1001).” 

? Action plans are created to provide LEAs with an organized means of identifying and addressing missing 
or incomplete data, as well as the necessary tools to ensure that the data they submit for future collections 
are true and correct. 
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collection; and scheduling conference calls with any LEA that requests similar or repetitious action plans 
over the course of two or more collections. 


GAO’s Fourth Recommendation 


GAO states that the Assistant Secretary for OCR “should correct current and previous data submissions 
indicating zero incidents of restraint and seclusion that it knows are incorrect.” 


OCR assumes that this recommendation centers on the 2015-16 collection, which closed on June 23, 2017. 
If this assumption is accurate, then GAO is aware that OCR has been actively working to improve data 
quality as part of the Department’s initiative on the inappropriate use of restraint and seclusion. Once OCR 
completes its review of responses to the letters it issued to recipients selected for data quality reviews, OCR 
will amend the 2015-16 data notes to ensure that the public is aware of any potential changes to the data.* 
However, GAO should know that it is not feasible to continuously update the published datafiles for closed 
collections beyond the normal corrections period, due to the need to finalize consistent data for the federal 
agencies, policymakers, researchers, educators, school officials, parents/guardians, students, other 
stakeholders, and members of the public who use CRDC data. For these reasons, to the extent the GAO 
recommendation goes beyond amending data notes, and suggests that OCR re-open final published datafiles 
for past collections, OCR does not agree with the recommendation. However, OCR will focus tts efforts 
on steps to improve the reporting of incidents of restraint and seclusion for future collections. 


On a final point, OCR believes it would be helpful to note that in the administration of the CRDC, OCR 
and the contractor work closely together, and OCR makes all final substantive decisions for the contractors 
to implement. In addition, when it comes to the provision of technical assistance to SEAs and LEAs, all 
novel or significant questions are brought to OCR for a response. 


As previously shared, OCR has made restraint and seclusion the focus of its primary proactive compliance 
initiative for this fiscal year. We will continue to take steps to improve the quality of data received from 
SEAs and LEAs. This issue is too important to expect anything less. 

The Department appreciates the opportunity to provide these comments, as well as the enclosed technical 
comments on the GAO draft report. We are available to respond to any questions or have further 


discussions if that would be helpful. 


Sincerely, 


Kenneth L. Marcus 
Assistant Secretary 
for Civil Rights 


Enclosure 


® https://ocrdata.ed. gov/Downloads/Data-Notes-2015-16-CRDC.pdf. 
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